{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "---\n",
    "sidebar_label: Ollama\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ChatOllama\n",
    "\n",
    "[Ollama](https://ollama.ai/) allows you to run open-source large language models, such as LLaMA2, locally.\n",
    "\n",
    "Ollama bundles model weights, configuration, and data into a single package, defined by a Modelfile. \n",
    "\n",
    "It optimizes setup and configuration details, including GPU usage.\n",
    "\n",
    "For a complete list of supported models and model variants, see the [Ollama model library](https://ollama.ai/library).\n",
    "\n",
    "## Setup\n",
    "\n",
    "First, follow [these instructions](https://github.com/jmorganca/ollama) to set up and run a local Ollama instance:\n",
    "\n",
    "* [Download](https://ollama.ai/download)\n",
    "* Fetch a model via `ollama pull <model family>`\n",
    "* e.g., for `Llama-7b`: `ollama pull llama2`\n",
    "* This will download the most basic version of the model (e.g., minimum # parameters and 4-bit quantization)\n",
    "* On Mac, it will download to:\n",
    "\n",
    "`~/.ollama/models/manifests/registry.ollama.ai/library/<model family>/latest`\n",
    "\n",
    "* And we can specify a particular version, e.g., for `ollama pull vicuna:13b-v1.5-16k-q4_0`\n",
    "* The file is here with the model version in place of `latest`\n",
    "\n",
    "`~/.ollama/models/manifests/registry.ollama.ai/library/vicuna/13b-v1.5-16k-q4_0`\n",
    "\n",
    "You can easily access models in a few ways:\n",
    "\n",
    "1/ if the app is running:\n",
    "\n",
    "* All of your local models are automatically served on `localhost:11434`\n",
    "* Select your model when setting `llm = Ollama(..., model=\"<model family>:<version>\")`\n",
    "* If you set `llm = Ollama(..., model=\"<model family\")` withoout a version it will simply look for `latest`\n",
    "\n",
    "2/ if building from source or just running the binary: \n",
    "\n",
    "* Then you must run `ollama serve`\n",
    "* All of your local models are automatically served on `localhost:11434`\n",
    "* Then, select as shown above\n",
    "\n",
    "\n",
    "## Usage\n",
    "\n",
    "You can see a full list of supported parameters on the [API reference page](https://api.python.langchain.com/en/latest/llms/langchain.llms.ollama.Ollama.html).\n",
    "\n",
    "If you are using a LLaMA `chat` model (e.g., `ollama pull llama2:7b-chat`) then you can use the `ChatOllama` interface.\n",
    "\n",
    "This includes [special tokens](https://huggingface.co/blog/llama2#how-to-prompt-llama-2) for system message and user input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.callbacks.manager import CallbackManager\n",
    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
    "from langchain.chat_models import ChatOllama\n",
    "\n",
    "chat_model = ChatOllama(\n",
    "    model=\"llama2:7b-chat\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Optionally, pass `StreamingStdOutCallbackHandler` to stream tokens:\n",
    "\n",
    "```\n",
    "chat_model = ChatOllama(\n",
    "    model=\"llama2:7b-chat\",\n",
    "    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='\\nArtificial intelligence (AI) has a rich and diverse history that spans several decades. Here is a brief overview of the major milestones and events in the development of AI:\\n\\n1. 1950s: The Dartmouth Conference: The field of AI was officially launched at a conference held at Dartmouth College in 1956. Attendees included computer scientists, mathematicians, and cognitive scientists who were interested in exploring the possibilities of creating machines that could simulate human intelligence.\\n2. 1951: The Turing Test: British mathematician Alan Turing proposed a test to measure a machine\\'s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. The Turing Test has since become a benchmark for measuring the success of AI systems.\\n3. 1956: The First AI Program: Computer scientist John McCarthy created the first AI program, called the Logical Theorist, which was designed to reason and solve problems using logical deduction.\\n4. 1960s: Rule-Based Expert Systems: The development of rule-based expert systems, which used a set of rules to reason and make decisions, marked a significant milestone in the history of AI. These systems were widely used in industries such as banking, healthcare, and transportation.\\n5. 1970s: Machine Learning: Machine learning, which enables machines to learn from data without being explicitly programmed, emerged as a major area of research in AI. This led to the development of algorithms such as decision trees and neural networks.\\n6. 1980s: Expert Systems: The development of expert systems, which were designed to mimic the decision-making abilities of human experts, reached its peak in the 1980s. These systems were widely used in industries such as banking and healthcare.\\n7. 1990s: AI Winter: Despite the progress that had been made in AI research, the field experienced a decline in funding and interest in the 1990s, which became known as the \"AI winter.\"\\n8. 2000s: Machine Learning Resurgence: The resurgence of machine learning, driven by advances in computational power and data storage, led to a new wave of AI research and applications.\\n9. 2010s: Deep Learning: The development of deep learning algorithms, which are capable of learning complex patterns in large datasets, marked a significant breakthrough in AI research. These algorithms have been used in applications such as image and speech recognition, natural language processing, and autonomous vehicles.\\n10. Present Day: AI is now being applied to a wide range of industries and domains, including healthcare, finance, transportation, and education. The field is continuing to evolve, with new technologies and applications emerging all the time.\\n\\nOverall, the history of AI reflects a long-standing interest in creating machines that can simulate human intelligence. While the field has experienced periods of progress and setbacks, it continues to evolve and expand into new areas of research and application.')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.schema import HumanMessage\n",
    "\n",
    "messages = [HumanMessage(content=\"Tell me about the history of AI\")]\n",
    "chat_model(messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extraction\n",
    " \n",
    "Update your version of Ollama and supply the [`format`](https://github.com/jmorganca/ollama/blob/main/docs/api.md#json-mode) flag.\n",
    "\n",
    "We can enforce the model to produce JSON.\n",
    "\n",
    "**Note:** You can also try out the experimental [OllamaFunctions](https://python.langchain.com/docs/integrations/chat/ollama_functions) wrapper for convenience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.callbacks.manager import CallbackManager\n",
    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
    "from langchain.chat_models import ChatOllama\n",
    "\n",
    "chat_model = ChatOllama(\n",
    "    model=\"llama2\",\n",
    "    format=\"json\",\n",
    "    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"morning\": {\"sky\": \"pink\", \"sun\": \"rise\"}, \"daytime\": {\"sky\": \"blue\", \"sun\": \"high\"}, \"afternoon\": {\"sky\": \"gray\", \"sun\": \"peak\"}, \"evening\": {\"sky\": \"orange\", \"sun\": \"set\"}}\n",
      " \t\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from langchain.schema import HumanMessage\n",
    "\n",
    "messages = [\n",
    "    HumanMessage(\n",
    "        content=\"What color is the sky at different times of the day? Respond using JSON\"\n",
    "    )\n",
    "]\n",
    "\n",
    "chat_model_response = chat_model(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"name\": \"John\",\n",
      "  \"age\": 35,\n",
      "  \"fav_food\": \"pizza\"\n",
      "}\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "from langchain.schema import HumanMessage\n",
    "\n",
    "json_schema = {\n",
    "    \"title\": \"Person\",\n",
    "    \"description\": \"Identifying information about a person.\",\n",
    "    \"type\": \"object\",\n",
    "    \"properties\": {\n",
    "        \"name\": {\"title\": \"Name\", \"description\": \"The person's name\", \"type\": \"string\"},\n",
    "        \"age\": {\"title\": \"Age\", \"description\": \"The person's age\", \"type\": \"integer\"},\n",
    "        \"fav_food\": {\n",
    "            \"title\": \"Fav Food\",\n",
    "            \"description\": \"The person's favorite food\",\n",
    "            \"type\": \"string\",\n",
    "        },\n",
    "    },\n",
    "    \"required\": [\"name\", \"age\"],\n",
    "}\n",
    "\n",
    "messages = [\n",
    "    HumanMessage(\n",
    "        content=\"Please tell me about a person using the following JSON schema:\"\n",
    "    ),\n",
    "    HumanMessage(content=json.dumps(json_schema, indent=2)),\n",
    "    HumanMessage(\n",
    "        content=\"Now, considering the schema, tell me about a person named John who is 35 years old and loves pizza.\"\n",
    "    ),\n",
    "]\n",
    "\n",
    "chat_model_response = chat_model(messages)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-modal\n",
    "\n",
    "Ollama has support for multi-modal LLMs, such as [bakllava](https://ollama.ai/library/bakllava) and [llava](https://ollama.ai/library/llava).\n",
    "\n",
    "Browse the full set of versions for models with `tags`, such as [here](https://ollama.ai/library/llava/tags).\n",
    "\n",
    "Download the desired LLM:\n",
    "```\n",
    "ollama pull bakllava\n",
    "```\n",
    "\n",
    "Be sure to update Ollama so that you have the most recent version to support multi-modal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%pip install pillow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<img src=\"\" />"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import base64\n",
    "from io import BytesIO\n",
    "\n",
    "from IPython.display import HTML, display\n",
    "from PIL import Image\n",
    "\n",
    "\n",
    "def convert_to_base64(pil_image):\n",
    "    \"\"\"\n",
    "    Convert PIL images to Base64 encoded strings\n",
    "\n",
    "    :param pil_image: PIL image\n",
    "    :return: Re-sized Base64 string\n",
    "    \"\"\"\n",
    "\n",
    "    buffered = BytesIO()\n",
    "    pil_image.save(buffered, format=\"JPEG\")  # You can change the format if needed\n",
    "    img_str = base64.b64encode(buffered.getvalue()).decode(\"utf-8\")\n",
    "    return img_str\n",
    "\n",
    "\n",
    "def plt_img_base64(img_base64):\n",
    "    \"\"\"\n",
    "    Disply base64 encoded string as image\n",
    "\n",
    "    :param img_base64:  Base64 string\n",
    "    \"\"\"\n",
    "    # Create an HTML img tag with the base64 string as the source\n",
    "    image_html = f'<img src=\"data:image/jpeg;base64,{img_base64}\" />'\n",
    "    # Display the image by rendering the HTML\n",
    "    display(HTML(image_html))\n",
    "\n",
    "\n",
    "file_path = \"/Users/rlm/Desktop/Eval_Sets/multi_modal_presentations/DDOG/img_23.jpg\"\n",
    "pil_image = Image.open(file_path)\n",
    "\n",
    "image_b64 = convert_to_base64(pil_image)\n",
    "plt_img_base64(image_b64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AIMessage(content='90%')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.chat_models import ChatOllama\n",
    "from langchain_core.messages import HumanMessage\n",
    "\n",
    "chat_model = ChatOllama(\n",
    "    model=\"bakllava\",\n",
    ")\n",
    "\n",
    "# Call the chat model with both messages and images\n",
    "content_parts = []\n",
    "image_part = {\n",
    "    \"type\": \"image_url\",\n",
    "    \"image_url\": f\"data:image/jpeg;base64,{image_b64}\",\n",
    "}\n",
    "text_part = {\"type\": \"text\", \"text\": \"What is the Daollar-based gross retention rate?\"}\n",
    "\n",
    "content_parts.append(image_part)\n",
    "content_parts.append(text_part)\n",
    "prompt = [HumanMessage(content=content_parts)]\n",
    "chat_model(prompt)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
